Image pickup system, image capturing method, and computer-readbale storage medium storing program for performing image capturing method

ABSTRACT

An image pickup system causes a first illumination unit to emit light toward a subject at a first angle without causing a second illumination unit for emitting light toward the subject at a second angle different from the first angle, and causes an image pickup unit to capture the image of the subject so as to obtain a first captured image. The image pickup system causes the second illumination unit to emit light without causing the first illumination unit to emit light, and causes the image pickup unit to capture the image of the subject so as to obtain a second captured image. The image pickup system associates the first captured image with the second captured image.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a technique for capturing an image of apartially lustrous subject.

2. Description of the Related Art

It is very difficult to capture images of metallic lustrous subjectssuch as folding screens (byobu) or sliding screens (fusuma) coated withgold leaf. Image capturing of such a metallic lustrous subject issimilar to image capturing of a mirror. Accordingly, if image capturingof a metallic lustrous subject is performed, a portion at which lightemitted from a light source is reflected is present in a captured imageof the subject. In this case, the color and gradation of the capturedimage of the subject differs from the actual color and gradation of thesubject.

For example, if light is emitted toward a metallic lustrous subject fromthe front side of the subject at the time of image capturing,high-intensity light specularly reflected from the subject enters acamera and a portion having a color of a light source is present in acaptured image of the subject. On the other hand, if light is emittedtoward a metallic lustrous subject from an oblique direction withrespect to the subject at the time of image capturing, the color of ametallic luminous surface of the subject can be accurately reproduced,but a gradation characteristic is reduced and a captured image of thesubject loses a metallic luster.

A method of capturing an image of a metallic lustrous subject using twolight sources has been proposed (see, Japanese Patent Laid-Open No.05-80394). Light emitted from one of these light sources is transmittedto the subject from the front side of the subject via a screen having awhite diffusion effect so as to illuminate the subject with white light.Strobe light emitted from the other one of these light sources isdirectly transmitted to the subject from an oblique direction withoutusing a screen. In order to reproduce the color, gradation, and lusterof the subject, lights transmitted from two directions are combined.

However, in such a method of combining lights transmitted from twodirections and performing image capturing using the combined light, ifthe positional relationship between these light sources or thecombination ratio between lights from these light sources varies, thestate of a metallic lustrous surface of the subject varies widely. Iflights transmitted from two directions are combined, the effects of thelights may cancel each other. Accordingly, if a desirable image cannotbe captured, it is required to repeatedly change the positionalrelationship between these light sources or the combination ratiobetween lights transmitted from these light sources until a desirableimage can be obtained.

SUMMARY OF THE INVENTION

The present invention provides an image pickup system and an imagecapturing method capable of reproducing the color and gradation of alustrous portion of a partially lustrous subject without requiring auser to adjust the balance between light sources at the time of imagecapturing of the subject.

According to an embodiment of the present invention, there is providedan image pickup system including: a first illumination unit configuredto emit light toward a subject at a first angle so as to illuminate thesubject; a second illumination unit configured to emit light toward thesubject at a second angle different from the first angle so as toilluminate the subject; an image pickup unit configured to capture animage of the subject; a control unit configured to cause only the firstillumination unit to emit light at the time of image capturing of thesubject performed by the image pickup unit so as to obtain a firstcaptured image, and cause only the second illumination unit to emitlight at the time of image capturing of the subject performed by theimage pickup unit so as to obtain a second captured image; and an imageprocessing unit configured to associate the first captured image withthe second captured image.

According to an embodiment of the present invention, there is providedan image capturing method using a first illumination unit configured toemit light toward a subject at a first angle so as to illuminate thesubject, a second illumination unit configured to emit light toward thesubject at a second angle different from the first angle so as toilluminate the subject, and an image pickup unit configured to capturean image of the subject. The image capturing method includes: causingonly the first illumination unit to emit light at the time of imagecapturing of the subject performed by the image pickup unit so as toobtain a first captured image; causing only the second illumination unitto emit light at the time of image capturing of the subject performed bythe image pickup unit so as to obtain a second captured image; andassociating the first captured image with the second captured image.

According to an embodiment of the present invention, there is provided acomputer-readable storage medium having stored thereon a program forcontrolling an image pickup system including a first illumination unitconfigured to emit light toward a subject at a first angle so as toilluminate the subject, a second illumination unit configured to emitlight toward the subject at a second angle different from the firstangle so as to illuminate the subject, and an image pickup unitconfigured to capture an image of the subject. The program includes:causing only the first illumination unit to emit light at the time ofimage capturing of the subject performed by the image pickup unit so asto obtain a first captured image; causing only the second illuminationunit to emit light at the time of image capturing of the subjectperformed by the image pickup unit so as to obtain a second capturedimage; and associating the first captured image with the second capturedimage.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B illustrate an exemplary configuration of an image pickupsystem.

FIG. 2 illustrates a digital camera and a pan head used in an imagepickup system.

FIGS. 3A, 3B, and 3C illustrate a divisional image capturing process.

FIG. 4 is a flowchart illustrating an image capturing process performedby an image pickup system according to an exemplary embodiment of thepresent invention.

FIG. 5 illustrates a process of combining pieces of obtained image data.

FIG. 6 illustrates a process of generating printing plates from imagedata.

FIG. 7 is a flowchart illustrating an image capturing process performedby an image pickup system according to another embodiment of the presentinvention.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present invention will be described in detail belowwith reference to the accompanying drawings.

FIGS. 1A and 1B illustrate a configuration of an image pickup systemaccording to an exemplary embodiment. An image pickup system accordingto this embodiment is suitable for image capturing of a partiallymetallic lustrous subject 301 such as a gilded or gold-painted foldingscreen (byobu) or sliding screen (fusuma).

Referring to FIGS. 1A and 1B, an image pickup apparatus (digital camera)100 for capturing an image of the subject 301 and a pan head 200 forrotating the digital camera 100 in pitch and yaw directions areillustrated.

Light sources 302 and 303 are used for emitting light toward the subject301 from a diagonally left or right direction with respect to thesubject 301 (from a point located so that an angle formed between animage capturing optical axis L and the line joining the point to theintersection of the image capturing optical axis L and the subject 301is an angle larger than 45°), and are disposed symmetrically withrespect to the image capturing optical axis L. In this embodiment, theimage capturing optical axis L means an image capturing optical axis ofthe digital camera 100 when the digital camera 100 faces the subject301, that is, the image capturing optical axis L is orthogonal to thesubject 301.

Light sources 304 and 305 for emitting light toward the subject 301 fromthe substantial front side of the subject 301 are symmetrically disposedwith respect to the image capturing optical axis L. White diffusionplates 306 and 307 for diffusing irradiated light are disposed in frontof the light sources 304 and 305, respectively, and are, for example,tracing paper.

A control unit 1 is configured to control the digital camera 100, thepan head 200, and the light sources 302 to 305. As illustrated in FIG.1A, the control unit 1 causes only the light sources 302 and 303 to emitlight and causes the digital camera 100 to perform image capturing ofthe subject 301, thus obtaining image data. As illustrated in FIG. 1B,the control unit 1 causes only the light sources 304 and 305 to emitlight and causes the digital camera 100 to perform image capturing ofthe subject 301, thus obtaining image data.

An image processing unit 2 associates the image data obtained by imagecapturing performed in the state illustrated in FIG. 1A with the imagedata obtained by image capturing performed in the state illustrated inFIG. 1B. More specifically, the image processing unit 2 combines thesepieces of image data so as to reproduce all of the color, gradation, andluster of the subject 301.

FIG. 2 illustrates the digital camera 100 and the pan head 200 used inan image pickup system according to this embodiment. The pan head 200 ismainly composed of a pitch rotation member 201 and a yaw rotation member202. The digital camera 100 is fixed on the pitch rotation member 201with a fixing screw 203.

The pitch rotation member 201 is coupled to the yaw rotation member 202via a rotating shaft 204 on the opposite sides of the pitch rotationmember 201. The pitch rotation member 201 can rotate about a rotationaxis a in a pitch direction (direction indicated by an arrow A). The yawrotation member 202 is provided with a motor 206 for rotating the pitchrotation member 201 in the pitch direction. The motor 206 is configuredto rotate the pitch rotation member 201 and fix the pitch rotationmember 201 at a position in the pitch direction with respect to the yawrotation member 202.

The yaw rotation member 202 is supported by a motor 207 at the bottomthereof, and can rotate about a rotation axis b in a yaw direction(direction indicated by an arrow B).

The pan head 200 capable of rotating about two axes is fixed to a tripod208, and can be moved to a predetermined position with respect to asubject.

In this embodiment, using the digital camera 100 and the pan head 200,divisional image capturing is performed. A divisional image capturingprocess will be described with reference to FIGS. 3A, 3B, and 3C. It isassumed that divisional image capturing of an image capturing subjectillustrated in FIG. 3A is performed using the digital camera 100 and thepan head 200 illustrated in FIG. 2.

In divisional image capturing, it is required to divide an imagecapturing area of the subject into sub-image capturing areas so thatportions of these sub-image capturing areas overlap each other. In thisembodiment, the image capturing area of the subject is divided intothree sub-image capturing areas in the horizontal direction and foursub-image capturing areas in the vertical direction, and a user performsthe divisional image capturing of these twelve sub-image capturing areasusing the digital camera 100 disposed at a normal position while movingthe pan head 200. As a result, as illustrated in FIG. 3B, twelve imagesare captured. In order to combine these twelve images (divided images)obtained by divisional image capturing, the overlapping portions aredetected. On the basis of the detected overlapping portions, thesetwelve images are combined. As a result, a combined image illustrated inFIG. 3C is generated.

If a subject has a metallic lustrous area having a reflectivity higherthan a predetermined reflectivity, it is required to consider how to uselight sources for illuminating the subject. If image capturing isperformed in a flash mode, a light incident direction to the subject isgenerally changed so as to prevent light emitted from a light sourcefrom being specularly reflected from the subject. As illustrated in FIG.1A, image capturing is performed using the light sources 302 and 303 foremitting light toward the subject 301 from a diagonally left or rightdirection with respect to the subject 301. As a result, the color of thesubject 301 can be accurately reproduced. However, in the case of ametallic lustrous subject, light transmitted from an oblique directionwith respect to the subject is not reflected from the subject to thedigital camera 100. Accordingly, a gradation characteristic is reduced,and a captured image of the subject loses the luster.

Therefore, as illustrated in FIG. 1B, by causing the light sources 304and 305 to emit light toward the subject 301 from the front side of thesubject 301, the light emitted from the light sources 304 and 305 isspecularly reflected from the subject 301. As a result, a gradationcharacteristic required for the reproduction of the luster of thesubject 301 can be obtained.

An image capturing process performed by an image pickup system accordingto this embodiment will be described with reference to FIGS. 4 and 5.FIG. 4 is a flowchart illustrating an image capturing process performedby an image pickup system according to this embodiment. FIG. 5illustrates a process of combining pieces of image data obtained in theimage capturing process illustrated in FIG. 4.

In step S101, an image pickup system causes the light sources 302 and303 for emitting light toward the subject 301 from a diagonally left orright direction with respect to the subject 301 to emit light insynchronization with each other (without causing the light sources 304and 305 to emit light), performs divisional image capturing, andcombines divided images obtained from the divisional image capturing. Asa result, captured image data A (501) illustrated in FIG. 5 is obtained.

In step S102, the image pickup system causes the light sources 304 and305 for emitting light toward the subject 301 from the front side of thesubject 301 to emit light in synchronization with each other (withoutcausing the light sources 302 and 303 to emit light), performsdivisional image capturing, and combines divided images obtained fromthe divisional image capturing. As a result, captured image data B (505)illustrated in FIG. 5 is obtained.

In step S103, the image pickup system divides the captured image data A(501) obtained in step S101 into metallic lustrous area (first area)image data 503 and non-metallic lustrous area (second area) image data502. In an image corresponding to the captured image data A (501), animage portion satisfying predetermined hue, saturation, and luminanceconditions is defined as a metallic lustrous area. More specifically,using the YUV space, an area defined as the metallic lustrous area isset. For example, in the UV space, if the horizontal axis denotes U andthe vertical axis denotes V, a hue of gold ranges from 150 degrees to180 degrees. As a luminance (Y) value of gold, a value equal to orlarger than 150/255 is set. Using the above-described values, themetallic lustrous area can be extracted. By masking a portioncorresponding to the metallic lustrous area image data 503 included inthe captured image data A (501), the non-metallic lustrous area imagedata 502 can be obtained.

In step S104, a metallic lustrous area color signal 504 is generatedfrom the metallic lustrous area image data 503 extracted in step S103.

In step S105, using the same method as that performed in steps S103 andS104, the captured image data B (505) obtained in step S102 is dividedinto a metallic lustrous area (first area) image data 506 and anon-metallic lustrous area (second area) image data 507. The imagepickup system generates a metallic lustrous area luminance signal 508from the metallic lustrous area image data 506. In step S105, themetallic lustrous area extraction, which has been performed upon thecaptured image data A, is performed again upon the captured image dataB. However, it may be determined whether there is a match between thecaptured image data A and the captured image data B, and on the basis ofthe determination result, divided areas obtained from the metalliclustrous area extraction performed upon the captured image data A may bechanged to be applied to the captured image data B.

In step S106, the image pickup system combines the metallic lustrousarea color signal 504 generated in step S104 from the captured imagedata A (501) and the metallic lustrous area luminance signal 508generated in step S105 from the captured image data B (505) so as togenerate image data 509 for the metallic lustrous area of the subject301.

In step S107, the image pickup system combines the non-metallic lustrousarea image data 502 generated in step S103 from the captured image dataA (501) and the image data 509 for the metallic lustrous area of thesubject 301 which has been generated in step S106 so as to generateimage data 510 of a final image.

As described previously, the image pickup system separately causes a setof the light sources 302 and 303 and a set of the light sources 304 and305 to emit light toward the partially metallic lustrous subject 301 atthe time of image capturing so as to generate characteristic signals,and combines these characteristic signals. As a result, the image pickupsystem can obtain appropriate image data by controlling the luminancesignal and color signal of a metallic lustrous surface without requiringa user to adjust the balance between these light sources at the time ofimage capturing.

The image data combination process performed at the time of imagecapturing has been described with reference to FIG. 5. If a particularcolor is used for the metallic lustrous area at the time of printing ofimage data, different plates may be used for the metallic lustrous areaand the non-metallic lustrous area. That is, as illustrated in FIG. 6,by masking a portion corresponding to the metallic lustrous area imagedata 503 included in the captured image data A (501), the non-metalliclustrous area image data 502 is obtained. A non-metallic lustrous areaplate 511 is generated from the non-metallic lustrous area image data502. By combining the metallic lustrous area color signal 504 generatedfrom the captured image A (501) and the metallic lustrous area luminancesignal 508 generated from the captured image B (505), a metalliclustrous area plate 512 is generated.

As described previously with reference to FIG. 2, if divisional imagecapturing is performed, it is required to combine divided images.Furthermore, in the present invention, it is required to combine twoimages so as to generate each type of signal. Accordingly, it isdesirable that an unfavorable factor in combining images at the time ofimage capturing be removed.

In the previous embodiment, after divisional image capturing of asubject has been performed using the light sources 302 and 303 on theleft and right sides of the subject, divisional image capturing of thesubject is performed using the light sources 304 and 305 on the frontside of the subject. In this case, at the time of the divisional imagecapturing using the light sources 302 and the 303 and the divisionalimage capturing using the light sources 304 and 305, the pan head 200repeatedly performs the same operation. Accordingly, if the sameoperation cannot be accurately repeated, the image pickup system may notobtain images captured at the same position and obtain an accurate imagecombination result.

In this embodiment, each time image capturing of each of sub-imagecapturing areas is performed, switching between the set of the lightsources 302 and 303 and the set of the light sources 304 and 305 isperformed.

FIG. 7 is a flowchart illustrating an image capturing process performedby an image pickup system according to this embodiment. In step S201,the image pickup system causes the light sources 302 and 303 foremitting light toward the subject 301 from an oblique direction withrespect to the subject 301 to emit light in synchronization with eachother (without causing the light sources 304 and 305 to emit light) atthe time of image capturing of one of the sub-image capturing areas.

In step S202, the image pickup system causes the light sources 304 and305 for emitting light toward the subject 301 from the front side of thesubject 301 to emit light in synchronization with each other (withoutcausing the light sources 302 and 303 to emit light) without changingthe position of the pan head 200, that is, at the same divisional imagecapturing point as that in step S201, at the time of image capturing ofthe above-described sub-image capturing area.

In step S203, the image pickup system determines whether image capturingof all of the sub-image capturing areas (twelve sub-image capturingareas in the exemplary case illustrated in FIG. 2) have been performed.If it is determined in step S203 that image capturing of all of thesub-image capturing areas have not yet been performed, the image pickupsystem moves the pan head 200 and repeats the process from step S201 tostep S202 at the next divisional image capturing point.

If it is determined in step S203 that image capturing of all of thesub-image capturing areas have already been performed, the image pickupsystem combines divided images obtained from the image capturingperformed in step S201 so as to obtain image data in step S204. As aresult, as illustrated in FIG. 5, the captured image data A (501) isobtained.

In step S205, the image pickup system combines divided images obtainedfrom the image capturing performed in step S202 so as to obtain imagedata. As a result, as illustrated in FIG. 5, the captured image data B(505) is obtained.

The process from step S206 to step S210 is the same as the process fromstep S103 to step S107 described previously in the previous embodiment.That is, in step S206, the captured image data A (501) obtained in stepS204 is divided into the metallic lustrous area image data 503 and thenon-metallic lustrous area image data 502.

In step S207, the metallic lustrous area color signal 504 is generatedfrom the metallic lustrous area image data 503 extracted in step S206.

In step S208, the captured image data B (505) obtained in step S205 isdivided into the metallic lustrous area image data 506 and thenon-metallic lustrous area image data 507. The metallic lustrous arealuminance signal 508 is generated from the metallic lustrous area imagedata 506.

In step S209, the image pickup system combines the metallic lustrousarea color signal 504 generated in step S207 from the captured imagedata A (501) and the metallic lustrous area luminance signal 508generated in step S208 from the captured image data B (505) so as togenerate the image data 509 for the metallic lustrous area of thesubject 301.

In step S210, the image pickup system combines the non-metallic lustrousarea image data 502 generated in step S206 from the captured image dataA (501) and the image data 509 generated in step S209 so as to generatethe image data 510 of a final image.

According to this embodiment, since the digital camera 100 performsimage capturing using the light sources 302 and 303 and image capturingusing the light sources 304 and 305 at the same position and combinesobtained pieces of captured image data, an accurate combination resultcan be obtained.

In the previous and present embodiments, it is assumed that the digitalcamera 100 is rotated by the pan head 200 in the pitch and yawdirections and performs divisional image capturing of a subject. In theabove description, the image capturing optical axis L means the imagecapturing optical axis of the digital camera 100 when the digital camera100 faces the subject 301. However, if the pan head 200 is rotated, theimage capturing optical axis L is not orthogonal to the surface of thesubject 301. Accordingly, even if the pan head 200 is rotated at thetime of divisional image capturing, it is desirable that the lightsources 302 and 303 be disposed so that an angle formed between theimage capturing optical axis L and the line joining the position of eachof the light sources 302 and 303 to the intersection of the imagecapturing optical axis L and the subject 301 is an angle larger than45°. In the above-described embodiments, although an exemplary case hasbeen described in which image capturing of a partially metallic lustroussubject is performed, the present invention is not limited thereto.Image capturing of a lustrous subject made of another type of materialsuch as specific resin or stone may be performed. In this case, forexample, like in the case of a metallic lustrous subject, an imageportion satisfying predetermined hue, saturation, and luminanceconditions may be specified as a lustrous area as described previouslywith reference to FIGS. 4 and 5.

The present invention may be achieved in such a manner that acomputer-readable storage medium storing program code (software) forimplementing the functions of the above-described embodiments issupplied to a system or an apparatus, and a computer (or a CPU or MPU)of the system or apparatus reads out the program code from the storagemedium and executes the read program code.

In this case, the program code read out from the storage medium achievesthe functions of the above-described embodiments. The program code isnot necessarily a single program. It may be multiple programs and mayinclude compiled code, interpreted code, scripts and the like.

As the storage medium for providing program code, for example, aflexible disk, a hard disk, an optical disc, a magneto-optical disk, aCD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, or a ROM maybe used.

The computer executes the read program code to achieve the functions ofthe above-described embodiments. Furthermore, an OS (basic system orOperating System) or the like running on the computer may perform partor all of actual processing under instructions of the program code toachieve the functions of the above-described embodiments.

Still furthermore, the program code read out from the storage medium maybe written to a memory provided in a function expansion board insertedinto the computer or in a function expansion unit connected to thecomputer. Subsequently, a CPU or the like provided in the functionexpansion board or the function expansion unit may perform part or allof actual processing under instructions of the program code so as toachieve the functions of the above-described embodiments.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Application No.2008-053628 filed Mar. 4, 2008, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image pickup system comprising: a firstillumination unit configured to emit light toward a subject at a firstangle so as to illuminate the subject; a second illumination unitconfigured to emit light toward the subject at a second angle differentfrom the first angle so as to illuminate the subject; an image pickupunit configured to capture an image of the subject; a control unitconfigured to cause only the first illumination unit to emit light atthe time of image capturing of the subject performed by the image pickupunit so as to obtain a first captured image, and cause only the secondillumination unit to emit light at the time of image capturing of thesubject performed by the image pickup unit so as to obtain a secondcaptured image; a division unit configured to divide each of the firstcaptured image and the second captured image into a first area and asecond area in accordance with at least any one of a hue condition, asaturation condition, and a luminance condition; and an image signalgeneration unit configured to generate an image signal for a specificarea of the subject using a luminance signal of the first area of thefirst captured image and a color signal of the first area of the secondcaptured image.
 2. The image pickup system according to claim 1, whereinthe image signal generation unit includes an image combination unitconfigured to combine an image signal of the second area extracted bythe division unit from the second captured image and the image signalfor the specific area of the subject which has been generated by theimage signal generation unit so as to generate a signal of a singleimage.
 3. The image pickup system according to claim 1, wherein thefirst area has a reflectivity higher than a predetermined reflectivity.4. The image pickup system according to claim 1, wherein the firstillumination unit emits light toward the subject from a front side ofthe subject, and the second illumination unit emits light toward thesubject from diagonally right and left directions with respect to thesubject.
 5. An image capturing method using a first illumination unitconfigured to emit light toward a subject at a first angle so as toilluminate the subject, a second illumination unit configured to emitlight toward the subject at a second angle different from the firstangle so as to illuminate the subject, and an image pickup unitconfigured to capture an image of the subject, the image capturingmethod comprising: causing only the first illumination unit to emitlight at the time of image capturing of the subject performed by theimage pickup unit so as to obtain a first captured image; causing onlythe second illumination unit to emit light at the time of imagecapturing of the subject performed by the image pickup unit so as toobtain a second captured image; dividing each of the first capturedimage and the second captured image into a first area and a second areain accordance with at least any one of a hue condition, a saturationcondition, and a luminance condition; and generating an image signal fora specific area of the subject using a luminance signal of the firstarea of the first captured image and a color signal of the first area ofthe second captured image.
 6. A computer-readable storage medium havingstored thereon a program for controlling an image pickup systemincluding a first illumination unit configured to emit light toward asubject at a first angle so as to illuminate the subject, a secondillumination unit configured to emit light toward the subject at asecond angle different from the first angle so as to illuminate thesubject, and an image pickup unit configured to capture an image of thesubject, the program stored on the computer-readable storage mediumcomprising: causing only the first illumination unit to emit light atthe time of image capturing of the subject performed by the image pickupunit so as to obtain a first captured image; causing only the secondillumination unit to emit light at the time of image capturing of thesubject performed by the image pickup unit so as to obtain a secondcaptured image; dividing each of the first captured image and the secondcaptured image into a first area and a second area in accordance with atleast any one of a hue condition, a saturation condition, and aluminance condition; and generating an image signal for a specific areaof the subject using a luminance signal of the first area of the firstcaptured image and a color signal of the first area of the secondcaptured image.